System, method and apparatus for macroscopic inspection of reflective specimens

ABSTRACT

An inspection apparatus includes a specimen stage, one or more imaging devices and a set of lights, all controllable by a control system. By translating or rotating the one or more imaging devices or specimen stage, the inspection apparatus can capture a first image of the specimen that includes a first imaging artifact to a first side of a reference point and then capture a second image of the specimen that includes a second imaging artifact to a second side of the reference point. The first and second imaging artifacts can be cropped from the first image and the second image respectively, and the first image and the second image can be digitally stitched together to generate a composite image of the specimen that lacks the first and second imaging artifacts.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of, and priority to, U.S.Provisional Application No. 62/883,931, filed Aug. 7, 2019, entitled“METHOD FOR IMAGING LARGE, REFLECTIVE SPECIMENS,” the contents of whichare incorporated by reference in their entirety. This application isfurther related to U.S. Pat. Application No. 16/262,017, filed Jan. 30,2019, entitled “MACRO INSPECTION SYSTEMS, APPARATUS AND METHODS,” thecontents of which are incorporated by reference in their entirety.

TECHNICAL FIELD

The present disclosure generally relates to macroscopic inspectionsystems, apparatus and methods for imaging reflective specimens.

BACKGROUND OF THE INVENTION

When performing macroscopic examination of a specimen (i.e., imaging, ina single field of view, a specimen, or an area of a specimen 200 mm orgreater) that is made of reflective material (e.g., glass, mirror,optical lenses, semiconductor wafers, etc.): (1) the reflection of animaging device positioned above the specimen can reflect off of thespecimen and appear in the image captured by the imaging device; and (2)illumination directed at the specimen can reflect off of the specimenand appear as hot spots in the captured image. These imaging artifacts(i.e., the reflection of the imaging device and/or the illumination hotspots) are undesirable.

FIG. 1A shows an illumination source 16 that provides an illuminationspace 18 that illuminates specimen S. Imaging device 10 and focusinglens 12 form imaging assembly 13 and define an imaging space 14 that iscaptured by imaging assembly 13. FIG. 1B shows an image tile 22 capturedby imaging assembly 13. Within image tile 22 is an image of a specimen24 and a darkspot 26 at the center. The darkspot can be a result ofimaging assembly 13′s reflection in the image or a shadow fromillumination 16.

Accordingly, it is desirable to provide a new mechanism for macroscopicexamination of a specimen that eliminates these undesirable imagingartifacts and can provide for multiple modes of illumination including,but not limited to brightfield, darkfield or oblique illumination;polarized light; cross-polarized light; and differential interferencecontrast (DIC), phase contrast. It is also desirable that each mode ofillumination provides variable illumination landscapes, as explainedherein, to detect features of a specimen. For purposes of thisspecification, macroscopic refers to an area approximately 0.5 cm² orgreater in a single field of view. Specimens as understood by a personof ordinary skill in the art refer to an article of examination (e.g., asemiconductor wafer or a biological slide), and features refer to knowncharacteristics of a specimen, as well as abnormalities and/or defects.Features can include but are not limited to: circuits, circuit boardcomponents, biological cells, tissue, defects (e.g., scratches, dust,fingerprints).

SUMMARY OF INVENTION

In one example, an inspection apparatus includes a specimen stage thatis configured to retain a specimen, one or more imaging devicespositioned above the specimen stage to capture images of the specimen, aset of lights on a platform between the specimen stage and the one ormore imaging devices, a control system coupled to the specimen stage,the one or more imaging devices, and the platform, where the controlsystem comprises one or more processors; and memory storing executableinstructions that, as a result of being executed by the one or moreprocessors, cause the control system to: provide first instructions tothe one or more imaging devices to capture a first image of thespecimen, the first image comprising a first imaging artifact to a firstside of a reference point, provide second instructions to the one ormore imaging devices to capture a second image of the specimen, thesecond image comprising a second imaging artifact to a second side ofthe reference point; crop the imaging artifact from the first image andthe second image, and to digitally stitch together the first image andthe second image to generate a composite image of the specimen, thecomposite image lacking the first imaging artifact and the secondimaging artifact.

In some examples, the the executable instructions further cause thecontrol system to translate the one or more imaging devices in a firstdirection to a first position above and to the first side of thereference point to capture the first image and to translate the one ormore imaging devices in a second direction to a second position aboveand to the second side of the reference point to allow the one or moreimaging devices to capture the second image.

In some examples, the executable instructions cause the control systemto translate the specimen stage in a first direction to a first positionunder and to the first side of the reference point to capture the firstimage of the specimen and to translate the specimen stage in a seconddirection to a second position under and to the second side of thereference point to capture the second image of the specimen.

In some examples, the reference point is positioned along a centerlineof the specimen.

In some examples, the specimen stage or the one or more imaging devicesare moveable along a rotational axis.

In some examples, the one or more imaging devices include a firstimaging device positioned above and to the first side of the referencepoint and a second imaging device positioned above and to the secondside of the reference point and the inspection apparatus furtherincludes an aperture slider positioned below the first imaging deviceand the second imaging device, the aperture slider comprising anaperture to allow capture of images of the specimen using either thefirst imaging device or the second imaging device.

In some examples, the executable instructions cause the control systemto translate the aperture slider to a first position such that theaperture is aligned with the first imaging device to capture the firstimage and to translate the aperture slider to a second position suchthat the aperture is aligned with the second imaging device to capturethe second image.

In some examples, the executable instructions cause the control systemto translate the platform, activate one or more combinations of the setof lights to determine an illumination profile, analyze the first imageof the specimen to identify a specimen classification, select, based onthe specimen classification, the illumination profile, and adjust theplatform and the set of lights according to the illumination profile.

In some examples, the inspection apparatus includes a barrier configuredto diffuse light reflected from the specimen retained on the specimenstage back onto the specimen.

In some examples, the executable instructions cause the control systemto compare a first overlap area of the first image to a second overlaparea of the second image to determine that a matching image has beenidentified to allow for digital stitching of the first image and thesecond image.

In one example, a method includes receiving a specimen on a specimenstage of an inspection apparatus, identifying a reference point of thespecimen, capturing a first image of the specimen that includes a firstimaging artifact to a first side of the reference point, capturing asecond image of the specimen that includes a second imaging artifact toa second side of the reference point, evaluating the second image of thespecimen to determine that the second image can be used with the firstimage, cropping the first imaging artifact from the first image and thesecond imaging artifact from the second image; and digitally stitchingtogether the first image and the second image to generate the compositeimage of the specimen, the composite image lacking the first imagingartifact and the second imaging artifact.

In some examples, the method further includes translating an imagingdevice of the inspection apparatus in a first direction to a firstposition above and to the first side of the reference point to capturethe first image, and translating the imaging device of the inspectionapparatus in a second direction to a second position above and to thesecond side of the reference point to capture the second image.

In some examples, the method further includes translating the specimenstage in a first direction to a first position under and to the firstside of the reference point to capture the first image, and translatingthe specimen stage to a second position in a second direction to aposition under and to the second side of the reference point to capturethe second image.

In some examples, the method further includes rotating the specimenstage to a first position to capture the first image, cropping the firstimage to remove a first portion of the first image that includes thefirst imaging artifact, rotating the specimen stage to a second positionto capture the second image, cropping the second image to remove asecond portion of the second image that includes the second imagingartifact, and digitally rotating the second image to initiate evaluationof the second image.

In some examples, the method further includes translating an apertureslider of the inspection apparatus in a first direction to position anaperture below a first imaging device of the inspection apparatus tocapture the first image, where the first imaging device is positionedabove and to the first side of the reference point, and translating theaperture slider of the inspection apparatus in a second direction toposition the aperture below a second imaging device of the inspectionapparatus to capture the second image, where the second imaging deviceis positioned above and to the second side of the reference point.

In some examples, the method further includes translating a platform ofthe inspection system, where a set of lights are disposed on theplatform, activating one or more combinations of the set of lights todetermine an illumination profile, analyzing the first image of thespecimen to identify a specimen classification, selecting, based on thespecimen classification, the illumination profile, and adjusting theplatform and the set of lights according to the illumination profile.

In some examples, the method further includes rotating an imaging deviceof the inspection apparatus in a first direction to position the imagingdevice to the first side of the reference point to capture the firstimage, and rotating the imaging device of the inspection apparatus in asecond direction to position the imaging device to the second side ofthe reference point to capture the second image.

In some examples, the method further includes diffusing light reflectedfrom the specimen retained on the specimen stage back onto the specimen.

In some examples, the method further includes comparing a first overlaparea of the first image to a second overlap area of the second image todetermine that a matching image has been identified to allow for digitalstitching of the first image and the second image.

In some examples, the specimen stage is moveable along an X axis, a Yaxis, a Z axis, and a rotational axis.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features of the disclosure can be obtained, a moreparticular description of the principles briefly described above will berendered by reference to specific embodiments thereof which areillustrated in the appended drawings. Understanding that these drawingsdepict only exemplary embodiments of the disclosure and are nottherefore to be considered to be limiting in their scope, the principlesherein are described and explained with additional specificity anddetail through the use of the accompanying drawings in which:

FIG. 1A shows an illumination source that provides an illumination spacethat illuminates a specimen;

FIG. 1B shows an image tile captured by an imaging assembly;

FIG. 2 shows an example of a macroscopic (macro) inspection systemaccording to some embodiments of the disclosed subject matter;

FIG. 3A shows a top view of a light ring assembly according to someembodiments of the disclosed subject matter;

FIG. 3B shows a side view of a light ring assembly according to someembodiments of the disclosed subject matter;

FIG. 4A shows a cone of illumination resulting from a maximum,unrestricted illumination beam for illumination of a specimen accordingto some embodiments of the disclosed subject matter;

FIG. 4B shows a minimization of a cone of illumination resulting frommovement of a light deflector to a first position according to someembodiments of the disclosed subject matter;

FIG. 4C shows a minimization of a cone of illumination resulting frommovement of a light deflector to a second position according to someembodiments of the disclosed subject matter;

FIG. 5A shows a macro inspection system comprising an imagingtranslation platform in a first position for capturing a first image ofa specimen according to some embodiments of the disclosed subjectmatter;

FIG. 5B shows a macro inspection system comprising an imagingtranslation platform translated to a second position for capturing asecond image of a specimen to create an artifact-free image of thespecimen according to some embodiments of the disclosed subject matter;

FIG. 6A shows a first image of a specimen captured at a first positionusing stage translation, imaging assembly platform translation oraperture translation according to some embodiments of the disclosedsubject matter;

FIG. 6B shows a second image of a specimen captured at a second positingusing stage translation, imaging assembly platform translation oraperture translation according to some embodiments of the disclosedsubject matter;

FIG. 6C shows an artifact-free image of a specimen created by stitchingthe first image of the specimen and the second image of the specimenaccording to some embodiments of the disclosed subject matter;

FIG. 7 shows an example imaging method for creating a compositeartifact-free image of a specimen by translating an imaging assemblyaccording to some embodiments of the disclosed subject matter;

FIG. 8 shows an example imaging method for creating a compositeartifact-free image of a specimen by translating a specimen stageaccording to some embodiments of the disclosed subject matter;

FIG. 9A shows a macro inspection system comprising two imagingassemblies and a translatable aperture slider for capturing a firstimage of a specimen at a first position according to some embodiments ofthe disclosed subject matter;

FIG. 9B shows a macro inspection system comprising two imagingassemblies and a translatable aperture slider for capturing a secondimage of a specimen at a second position according to some embodimentsof the disclosed subject matter;

FIG. 10 shows an example imaging method for creating a compositeartifact-free image of a specimen using two imaging assemblies and atranslatable aperture slider according to some embodiments of thedisclosed subject matter;

FIG. 11 shows a macro inspection system comprising a specimen stage andan imaging assembly that either or both can be rotated for creating anartifact-free image according to some embodiments of the disclosedsubject matter;

FIG. 12 shows a method for creating a composite artifact-free image of aspecimen by rotating a specimen stage of a macro inspection systemaccording to some embodiments of the disclosed subject matter;

FIG. 13A shows an example image captured at an initial position of aspecimen stage according to some embodiments of the disclosed subjectmatter;

FIG. 13B shows an example image wherein a portion of the image thatincludes an imaging artifact is cropped out of the image according tosome embodiments of the disclosed subject matter;

FIG. 13C shows an example image captured at a second position viarotation of the specimen stage of a macro inspection system according tosome embodiments of the disclosed subject matter;

FIG. 13D shows an example image wherein a portion of the image thatincludes an imaging artifact is cropped out of the image according tosome embodiments of the disclosed subject matter;

FIG. 13E shows an example of a cropped image being digitally rotated toan original position according to some embodiments of the disclosedsubject matter;

FIG. 13F shows an example of a cropped image digitally rotated to anoriginal position according to some embodiments of the disclosed subjectmatter;

FIG. 13G shows a composite artifact-free image of a specimen generatedby stitching two cropped images together according to some embodimentsof the disclosed subject matter;

FIG. 14 shows an example calibration method for calibrating macroinspection system to achieve different illumination landscapes accordingto some embodiments of the disclosed subject matter;

FIG. 15A shows an example method for illuminating a specimen using amacro system to achieve a desired illumination landscape according tosome embodiments of the disclosed subject matter;

FIG. 15B shows an example method for identifying a specimenclassification and automatically adjusting an illumination landscape ofthe macro inspection system according to some embodiments of thedisclosed subject matter;

FIG. 16 shows the general configuration of an embodiment of computeranalysis system according to some embodiments of the disclosed subjectmatter; and

FIG. 17 shows an example of using training data to train one or moreartificial intelligence algorithms that can be used on a receivedspecimen scan to create one or more illumination profiles for eachreceived specimen image according to some embodiments of the disclosedsubject matter.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

In accordance with some embodiments of the disclosed subject matter,mechanisms (which can include systems, methods, devices, apparatuses,etc.) for macroscopic examination of reflective specimens are provided.Macroscopic examination (sometimes referred to as inspection) refers toscanning, imaging, analyzing, measuring and any other suitable review ofa specimen using the disclosed macroscopic inspection mechanism. Thedisclosed macroscopic inspection mechanism includes one or more modes ofillumination that can each provide variable illumination landscapes, asdescribed herein. Although the following description refers tocomponents and methods implemented in a macroscopic inspectionmechanism, the components and methods described herein can also beimplemented in a microscope inspection system.

FIG. 2 illustrates an example of a macroscopic (macro) inspection system100 according to some embodiments of the disclosed subject matter. At ahigh level, the basic components of macro inspection system 100,according to some embodiments, include an illumination assembly (e.g.,light ring assembly 80) for providing light to a specimen S, a focusinglens 34, an imaging device 32, a specimen stage 50, a control system 70comprising hardware (e.g., one or more processors configured to performoperations described herein, etc.), software, and/or firmware and acomputer analysis system 75. Macro inspection system 100 can beimplemented as part of an optical inspection system that usestransmitted or reflected light.

In some embodiments, as shown in FIGS. 2, 3A and 3B, a light ringassembly 80 can be used as an illumination assembly for macro inspectionsystem 100 to provide light to a specimen (as represented byillumination space 90). One or more individual lights (e.g., LED lightsL1 to Ln) can be mounted to light ring assembly 80. Individual lights L1to Ln can be based on any type of suitable lighting technology,including but not limited to: light emitting diode (LED), organic lightemitting diode (OLED), fluorescent, halogen, incandescent, fiber optic,gas-plasma, cathode ray tube (CRT), liquid crystal display (LCD), laser,etc. Each light can be individually addressed. In further embodimens,the individual lights can be divided into sections (e.g., by position onthe light ring assembly, such as front, back, right, left), and eachsection can be addressable. Software, hardware and/or firmware (e.g.,control system 70) can control the activation, intensity and/or color ofeach light or section by its address. For instance, the control system70 an comprise one or more processors and memory that storesinstructions that, as a result of being executed by the one or moreprocessors, cause the control system 70 to perform the operationsdescribed herein. In some instances, the control system 70 isimplemented as an application or as a stand-alone computer system thatperforms the operations described herein. Activation refers to theturning on of a light, intensity refers to the rate at which lightenergy is delivered to a unit of surface, and color refers to an RGB(red, green, blue) color value, for example, where each color value isspecified as an integer from 0 to 255 based on 8-bit color, for example.Intensity can be determined by light meters, image sensors and/or othersuitable intensity measurement devices. Plurality of lights L1 to Ln canbe comprised of lights that project monochromatic, polychromatic and/orany combination thereof.

Each light L1 to Ln can provide oblique lighting at different angles ofincidence, from multiple directions, in accordance with some embodimentsof the disclosed subject matter. Three methods for varying the angles ofillumination, as described herein, include: (1) changing the angle of alight mounted to light ring assembly 80; (2) raising or lowering lightring assembly 80 in a z direction; and/or (3) positioning a lightdeflector such that part of the illumination beam from a light isblocked.

In one embodiment, each light can be mounted to light ring assembly 80at a desired angle relative to the specimen plane of a specimen whenretained on specimen stage 50. In further embodiments, each light’sangle can be controlled manually, or automatically by software,hardware, and/ or firmware (e.g., control system 70). A light’s anglecan be controlled individually or concurrently with one or more otherlights. Each light can be angled the same or different amounts.

In some embodiments, light ring assembly 80 can be configured so that itis movable along guiderails 48 of macro inspection system 100. In oneexample, light ring assembly 80 can be attached to guiderails 48 withsupport rods 81 a and 81 b and bearings 82 a and 82 b (as shown in FIG.3A). Note, that the illumination assembly is not limited to a ringformation. For example, other types of light formations are possible asdescribed in U.S. Pat. Application No. 16/262,017 entitled “MacroInspection Systems, Apparatus and Methods,” which is hereby incorporatedby reference herein in its entirety. Further, the movement of light ringassembly 80 to different positions along the height of the guiderails 48can be controlled manually, or automatically by software, hardware, and/or firmware (e.g., control system 70). Depending on its height inrelation to specimen stage 50, light ring assembly 80 can be used toprovide oblique or darkfield illumination to a specimen when retained onspecimen stage 50. For example, to provide variable angles of obliqueillumination, light ring assembly 80 can be positioned so that its lightcan be projected at different heights above a specimen plane (i.e., thetop planar surface of a specimen when positioned on specimen stage 50).In some embodiments, the specimen plane corresponds with a focal planeof macro inspection system 100 (i.e., the plane where the specimen is infocus). In further examples, to provide darkfield illumination, lightring assembly 80 can be positioned so that its light can be projected atthe same, or substantially the same, level of the specimen plane of aspecimen on specimen stage 50 to provide darkfield illumination to aspecimen when retained on specimen stage 50.

As used herein: oblique illumination refers to light projected towardthe specimen at an angle of incidence less than 90 degrees and greaterthan 0 degrees, typically greater than 1 degrees; darkfield illuminationrefers to light projected toward the specimen at an angle of incidenceless than 1 degrees and typically 0 degrees; and brightfieldillumination refers to light projected toward the specimen at an angleof incidence perpendicular (90 degrees) to the plane of the specimen.Brightfield illumination can refer to a light source that providesillumination through lens 34.

Depending on its distance (d) in relation to specimen stage 50, lightring assembly 80 can be used to provide oblique or darkfieldillumination to a specimen when retained on specimen stage 50. Providingoblique and darkfield lighting is described in U.S. Patent ApplicationNo. 16/262,017 entitled “Macro Inspection Systems, Apparatus andMethods,” which is hereby incorporated by reference herein in itsentirety. In some embodiments, light ring assembly 80 can be positionedso that illumination from the assembly is substantially parallel to aspecimen plane to provide darkfield illumination to a specimen whenretained on specimen stage 50. Substantially parallel is to beunderstood as having an angle of incidence from -1° to +1°, to allow forimperfections in alignment, but in some embodiments, the illuminationwill be on plane, i.e., at a d of 0, whereby illumination will bereflected only if there are features extending off of a perfectly flatplanar surface of a specimen. If a specimen is perfectly flat andfeatureless, then it would not reflect any of the substantially parallelillumination to lens 34, and such a specimen viewed by lens 34 will notbe illuminated. If there are protruding imperfections or other features,then the illumination from light ring assembly 80 will reflect off ofsuch imperfections and/or features and will be captured by imagingdevice 32 via lens 34. If its distance from specimen stage 50 is greaterthan 0, then light ring assembly 80 can be used to provide obliqueillumination to a specimen when retained on specimen stage 50.

As shown in FIGS. 4A-4C, light deflector 83 can be used for adjustingthe cone of illumination for each light Li. Each light Li can be mountedat an angle α from light ring assembly 80. Light ring assembly 80 can bepositioned perpendicular to specimen stage 50, and lights can be mountedat an angle α from 0 to 90 degrees, typically between 0 to 60 degrees(e.g., 10 degrees). A typical LED light can have a cone of illuminationof approximately 120 degrees. The cone of illumination is represented byany two vectors a, b, c, d, e, f and g. Vectors a and g represent themaximum, unrestricted illumination beam as seen in FIG. 4A. The cone ofillumination of the unrestricted light is represented by θ1. As shown,for example, in FIGS. 4B and 4C, light deflector 83 can be positionedover the lights, to minimize the cone of illumination. FIG. 4B showslight deflector 83 moved to a first position to block vector a(represented by a dotted line) and allow light vectors b through g tocontinue. The cone of illumination of this restricted light isrepresented by θ2, and has a smaller cone than θ1. FIG. 4C shows lightdeflector 83 moved to a second position, blocking vectors a, b and c(represented by dotted lines) and allowing vectors d, e, f and g tocontinue. The cone of illumination of this restricted light is θ3, andits cone of illumination is smaller than both θ1 and θ2. α and θ can beadjusted to provide an angle of illumination that illuminates thespecimen to show specific specimen features. For example, higher anglesof illumination are generally better for defining edges, while lowerangles of illumination are generally better for defining bumps. In someembodiments, a single light deflector is used to control the cone ofillumination for all the lights, and in other embodiments, individuallight deflectors can be used to control the cone of illumination foreach light. The cone of illumination for each light can be the same ordifferent.

While deflector 83, as shown in FIGS. 4A-4C, is lowered from top tobottom, it can also be configured to move from bottom to top, or in bothdirections. Regardless of the features that are being examined,deflector 83 can be positioned to prevent light from being directed tothe imaging device and causing imaging artifacts in captured images. Insome embodiments, deflector 83 can be positioned to direct light only toa specimen. In embodiments where a dome is included in macro inspectionsystem 100, deflector 83 can be adjusted to deflect light to the dome,to the specimen, and/or to the dome and the specimen. Single orindividual light deflectors 83 can be controlled manually, orautomatically by software, hardware, and/ or firmware (e.g., controlsystem 70). Adjustor screw 86, as shown in FIGS. 4A-4C, is one exampleof a mechanism that can be used to adjust light deflector 83.

In some embodiments, an XYZθ translation stage can be used for specimenstage 50. Specimen stage 50 can be driven by stepper motor, servermotor, linear motor, piezo motor, and/or any other suitable mechanism,including a manual mechanism. Specimen stage 50 can be configured tomove an object in the X axis, Y axis, Z axis and/or θ directionsmanually and/or under the control of any suitable controller (e.g.,control system 70). An actuator (e.g. actuator 39) can be used to makecoarse focus adjustments of, for example, 0 to 5 mm, 0 to 10 mm, 0 to 30mm, and/or any other suitable range(s) of distances. An actuator canalso be used in some embodiments to provide fine focus of, for example,0 to 50 µm, 0 to 100 µm, 0 to 200 µm, and/or any other suitable range(s)of distances. A person of skill in the art would understand that XYZθtranslation stage is just an example, and other suitable stages can beused (e.g., an XYZ translation stage, a θ translation stage, a Ztranslation stage).

In some embodiments, lens 34 and imaging device 32, which together formimaging assembly 33, can be supported on a translation assembly abovespecimen stage 50. Translation assembly includes imaging translationplatform 44, which can be configured to move imaging assembly 33 in theX axis, Y axis and/or θ directions manually and/or under the control ofany suitable controller (e.g., control system 70). Upper support frame46 can also include limits (e.g, left and right limits 43 a and 43 b)and/or encoder 45 for aligning imaging translation platform 44. Thelimits can be physical stops or switches (optical, mechanical,electronic or other) to indicate proper alignment of imaging translationplatform 44. The switches can be controlled by control system 70 to onlyallow images to be taken when imaging translation platform 44 is in aparticular alignment or to automatically capture images when the limitswitches are activated (e.g., by positioning imaging translationplatform 44 within limits 43 a and 43 b). Encoder 45 can be used to moreprecisely indicate the position of plaform 44, and can be used to onlyallow imaging or to automatically trigger imaging when platform 44 is ata particular position. In some embodiments, upper support frame 46 canbe configured so that it is movable along guiderails 48 in the Z axisdirection. To adjust focus, upper support frame 46 can be lowered orraised, bringing imaging assembly 33, which is coupled to support frame46, closer to or farther apart from specimen stage 50. Further, themovement of upper support frame 46 to different positions along theheight of the guiderails 48 can be controlled manually, or automaticallyby software, hardware, and/ or firmware (e.g., control system 70). Inother embodiments, the imaging device can be mounted direcly to uppersupport frame 46, and be translatable in the X axis, Y axis, Z axisand/or θ directions in a similar manner.

Lens 34 can have different magnification powers, and/or be configured tooperate with brightfield, darkfield or oblique illumination, polarizedlight, cross-polarized light, differential interference contrast (DIC),phase contrast and/or any other suitable form of illumination. The typeof lens used for macro inspection system 100 can be based on desiredcharacteristics, for example, field of view, numerical aperture, amongothers. In some embodiments, lens 34 can be a macro lens that can beused to view a specimen within a single field of view. Note, the termfield of view as understood by a person of ordinary skill in the artrefers to an area of examination that is captured at once by an imagesensor.

The illumination of a specimen on specimen stage 50 reflects up to lens34 mounted to an imaging device 32 (e.g., a camera), and imaging device32 can capture images and/or video of a specimen in imaging space 92. Insome embodiments, imaging device 32 can be a rotatable camera thatincludes an image sensor, configured to allow the camera to be alignedto a specimen, a stage and/or a feature on a specimen. The image sensorcan be, for example, a charged-coupled device (CCD), a complementarymetal-oxide semiconductor (CMOS) image sensor, and/or any other suitableelectronic device that converts light into one or more electricalsignals. Such electrical signals can be used to form images and/or videoof an object. In some embodiments, such electrical signals aretransmitted for display on a display screen connected to macroinspection system 100. Some example methods for rotating a camera thatcan be used by macro inspection system 100 are described in U.S. Pat.No. 10,048,477 entitled “Camera and Object Alignment to Facilitate LargeArea Imaging in Microscopy,” which is hereby incorporated by referenceherein in its entirety.

In some embodiments, macro inspection system 100 can include a barrier,for example dome 42, as shown in FIG. 2 , configured to reflect lightthat is reflected from the specimen back toward the specimen. Dome 42diffuses the light that it reflects back to the specimen to provide amore uniform illumination field. Although a dome-shaped barrier isshown, other shaped barriers, including: sphere, cone, cube or any threedimensional poly-sided shape such as a rhombohedron, can be used toprovide different angles of diffusion. In some embodiments, a lightdiffuser is formed in the shape of a dome or other shaped barrier, andin other embodiments, the dome, or other barrier, can be any material,but painted with light diffusing paint. Dome 42 can be coupled toimaging assembly 33, so that when imaging assembly 33 is moved, dome 42moves along with it.

A single light or multiple lights can be activated to illuminate aportion or an entire field of view at the specimen plane. The type ofspecimen being examined, the type of feature being examined, a region ofinterest on a specimen, and/or any other suitable criteria, candetermine which lights are activated and at what color and/or intensity.Further, software, hardware and/or firmware (e.g., control system 70)can control the angle of each individual light or concurrently with oneor more other lights. In some embodiments, the angles can be changedmanually. Each light can be angled the same or different amounts. Insome embodiments, light is not directed at dome 42, but at the specimen,and is reflected off of dome 42 back to the specimen in a more diffusemanner.

Each individual light can individually or together emit a vector oflight to illuminate a particular area on the specimen plane )″area ofillumination″(. The magnitude of this area of illumination can vary fromilluminating a portion of the specimen to encompassing the entirespecimen plane. The area of illumination can be calculated at differentaxial locations above, below or on the specimen plane (e.g., at the topof specimen stage 50, at the top of the specimen plane, at the focalplane, etc.) along the beam of light represented by the vectors. Theareas covered by each vector of light can either be overlapping in partwith the areas covered by the vector of light emitted from a neighboringlight bar or not overlapping at all. In some embodiments, one or morefocusing lenses and/or collimating lenses can be used to focus the areaof each light vector to a region suitable for a specimen on specimenstage 50.

In some embodiments, a single illumination vector ranges from 1 degreeor more to 180 degrees or less (60 or more to 10,800 or less minutes ofarc). In other embodiments, a single illumination vector ranges from 45degrees or more to 120 degrees or less (2,700 or more to 7,200 or lessminutes of arc), in other embodiments, from 30 degrees or more to 45degrees or less (1,800 or more to 2,700 or less minutes of arc), inother embodiments, from 10 degrees or more to 30 degrees or less (600 ormore to 1,800 or less minutes of arc), in other embodiments, from 5degrees or more to 10 degrees or less (300 or more to 600 or lessminutes of arc), and, in other embodiments, from 2 degrees or more to 5degrees or less (120 or more to 300 or less minutes of arc). The vectordepends upon the number and position of activated lights of lightassembly ring 80 relative to the position of the specimen.

Light ring assembly 80 can vary as to the number of lights, the size ofeach individual light, the cone angle of each individual light, thepitch (p) between lights and the distance between the lights and thearea where the light is projected. In some embodiments, the size ofspecimen stage 50, the specifications of lens 34, the size and/or typeof specimen being inspected, and/or the features of a specimen that arebeing examined, can determine the configuration of lights on lightassembly ring 80, including, for example, the arrangement of lights(whether in a ring or in other arrangements), the total number oflights, the distance, and/or the pitch (p).

As should be generally appreciated from the examples of illumination inFIGS. 4A-4C, the various embodiments of the present invention allow fordarkfield illumination, illumination at variable oblique angles andbrightfield illumination.

In some embodiments, control system 70 includes a controller andcontroller interface, and can control any settings of macro inspectionsystem 100 (e.g., intensity of lights, color of lights, turning on andoff one or more lights, pivoting or other movement of one or more lights(e.g., changing a light’s angle), movement of light ring assembly 80(e.g., in a z direction), movement of imaging platform 44; movement ofspecimen stage 50 or 150 (in x, y, Θ, and/or z directions), movement oflens 34 (in x, y, ⊖, and/or z directions), movement of imagingtranslation platform 44, recording of image data by imaging assembly 33,rotation or movement of imaging assembly 33, processing of illuminationdata, processing of image data). Control system 70 and applicablecomputing systems and components described herein can include anysuitable hardware (which can execute software in some embodiments), suchas, for example, computers, microprocessors, microcontrollers,application specific integrated circuits (ASICs), field-programmablegate arrays (FPGAs) and digital signal processors (DSPs) (any of whichcan be referred to as a hardware processor), encoders, circuitry to readencoders, memory devices (including one or more EPROMS, one or moreEEPROMs, dynamic random access memory )“DRAM”(, static random accessmemory )“SRAM”(, and/or flash memory), and/or any other suitablehardware elements. In some embodiments, individual components withinmacro inspection system 100 can include their own software, firmware,and/or hardware to control the individual components and communicatewith other components in macro inspection system 100.

In some embodiments, communication between the control system (e.g., thecontroller and controller interface) and the components of macroinspection system 100 can use any suitable communication technologies,such as analog technologies (e.g., relay logic), digital technologies(e.g., RS232, ethernet, or wireless), network technologies (e.g., localarea network (LAN), a wide area network (WAN), the Internet, Bluetoothtechnologies, Near-field communication technologies, Secure RFtechnologies, and/or any other suitable communication technologies.

In some embodiments, operator inputs can be communicated to controlsystem 70 using any suitable input device (e.g., keyboard, mouse,joystick, touch).

In some embodiments, control system 70 controls the activation,intensity and/or color of one or more of the plurality of lights, aswell as the position of lights L1 to Ln and/or light ring assembly 80(e.g., by adjusting the light ring assembly’s height, or by pivoting alight) to provide for variable illumination landscapes on a specimenwhen it is placed on specimen stage 50. Illumination landscape refers tothe color and/or intensity of light on a region of interest of aspecimen as a result of the activation and distribution of light fromthe one or more of the plurality of lights that is directed towards aspecimen. The illumination landscape can affect the image viewed throughlens 34 and/or images captured by imaging device 32. Control system 70can control the intensity of one or more of the plurality of lights toprovide a desired illumination landscape on a specimen plane and/orspecimen stage 50. For example, control system 70 can control theintensity of one or more of the plurality of lights to provide anillumination landscape of uniform intensity on a specimen plane and/orspecimen stage 50. The type of illumination landscape provided can bedetermined by the specimen type, mechanical and/or physical propertiesof a specimen (e.g., specimen size, specimen reflectivity), a specimenfeature being examined, a particular stage of a manufacturing and/orexamining process, or some other suitable variable, individually or inany combination thereof.

In some embodiments, computer analysis system 75 can be coupled to, orincluded in, macro inspection system 100 in any suitable manner usingany suitable communication technology, such as analog technologies(e.g., relay logic), digital technologies (e.g., RS232, ethernet, orwireless), network technologies (e.g., local area network (LAN), a widearea network (WAN), the Internet) Bluetooth technologies, Near-fieldcommunication technologies, Secure RF technologies, and/or any othersuitable communication technologies. Computer analysis system 75, andthe modules within computer analysis system 75, can be configured toperform a number of functions described further herein using imagesoutput by macro inspection system 100 and/or stored by computer readablemedia.

Computer analysis system 75 can include any suitable hardware (which canexecute software in some embodiments), such as, for example, computers,microprocessors, microcontrollers, application specific integratedcircuits (ASICs), field-programmable gate arrays (FPGAs), and digitalsignal processors (DSPs) (any of which can be referred to as a hardwareprocessor), encoders, circuitry to read encoders, memory devices(including one or more EPROMS, one or more EEPROMs, dynamic randomaccess memory (“DRAM”), static random access memory (“SRAM”), and/orflash memory), and/or any other suitable hardware elements.

It should be noted that while the control system 70 and the computeranalysis system 75 are illustrated in FIG. 2 as being separatecomponents of the macro inspection system 100, other implementations ofthe control system 70 and the computer analysis system 75 are within thescope of the present disclosure. For instance, in an embodiment, thecomputer analysis system 75 is implemented as an application or otherexecutable process of the control system 70. Further, while the computeranalysis system 75 is illustrated as being a component of the macroinspection system 100, the computer analysis system 75 can beimplemented as a separate system accessed over a communications network,such as the Internet or other network.

Computer-readable media can be any non-transitory media that can beaccessed by the computer and includes both volatile and nonvolatilemedia, removable and non-removable media. By way of example, and notlimitation, computer readable media can comprise computer storage mediaand communication media. Computer storage media can include volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital video disk (DVD) orother optical disk storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich can be used to store the desired information and which can beaccessed by the computer.

FIGS. 5A and 5B, along with FIGS. 6A-6C, show embodiments of macroinspection system 100 and an imaging method for creating anartifact-free image of a specimen by translating imaging translationplatform 44 to the right and left of a specimen centerline A2, whenplaced on stage 50. By translating imaging translation platform 44,optical centerline A1 of imaging assembly 33 can be offset from thecenterline of a specimen A2 that is placed on stage 50. The offsetamount must be sufficient, so that the entire imaging artifact, or theentire imaging artifact and some additional space, will appear in acaptured image on one side of specimen centerline A2 (e.g., as shown inFIGS. 6A and 6B). Note, the size of the offset of A1 of the imageassembly from centerline A2 (so that at least the entire artifactappears on one side of centerline A2) is typically equal to the size ofthe artifact as it appears in the captured image.

All references to moving or tranlating imaging assembly 33 in thisapplication refers to moving or translating imaging assembly 33 via atranslation mechanism (e.g., imaging translation platform 44). Also thefigures herein refer to using a centerline A1 or A2, there are justexample reference points, other reference points can be used to capturearticle-free images.

FIGS. 6A, 6B and 6C show images that can be captured under threedifferent modes: stage translation, imaging assembly platformtranslation or aperture translation.

Artifact-free as used herein refers to an image that does not includethe reflection of an imaging assembly and/or illumination hot spots.Note, embodiments of macro inspection system 100, as shown in FIGS. 5Aand 5B, do not show imaging space 92, or illumination space 90, but theconcepts of imaging and illumination spaces as shown in FIG. 2 apply toall embodiments.

As shown in FIGS. 5A and 5B, imaging translation platform can betranslated between two positions: to the right and to the left ofspecimen centerline A2. In FIG. 5A, imaging translation platform 44 ispositioned to the left of centerline A2, so that when imaging assembly33 captures an image of a specimen at that position, an imaging artifactfrom the reflection of imaging assembly 33 does not appear to the rightof centerline A2. FIG. 6A illustrates an example image 105 that can becaptured when imaging assembly is positioned to the left of centerlineA2 (as shown in FIG. 5A). As captured in image 105, specimen features X1and X2 appear on the left and right sides of the centerline A2. Imagingartifact 26, representing the reflection of the imaging assemblycaptured in the image, appears on the left side (i.e., the same sidethat imaging assembly 33 was positioned when the image was captured).The portion of the image that is artifact-free is indicated by boundingbox 114 (coincident with lines R1 on the right and R2 on the left).

In FIG. 5B, imaging translation platform 44 is positioned on theopposite side, to the right of centerline A2, so that when imagingassembly 33 captures an image of the specimen at that position, animaging artifact from the reflection of imaging assembly 33 does notappear to the left of centerline A2. FIG. 6B illustrates an exampleimage 106 that can be captured when imaging assembly is positioned tothe right of centerline A2 (as shown in FIG. 5B). Imaging artifact 26appears on the right side of image 106 (i.e., the same side that imagingassembly 33 was positioned when the image was captured). The portion ofthe image that is artifact-free is indicated by bounding box 114(coincident with lines R3 on the right and R4 on the left).

FIG. 7 shows an example imaging method 700 for creating a compositeartifact-free image of a specimen using embodiments of macro inspectionsystem 100, as shown in FIGS. 5A and 5B.

At 710, macro inspection system 100 can be initialized by adjusting theparameters that are specific to the macro inspection tool, and not tothe specimen being examined. Such parameters can include, but are notlimited to, focus, magnification, DIC prism, field of view. Focus ofinspection system 100 can be calibrated by using a sample specimen todetermine a suitable distance between imaging assembly 33 and specimenstage 50, and moving imaging assembly 33 or specimen stage 50, or both,farther apart or closer together until the desired focus is achieved.Focus can be controlled manually, or automatically by software,hardware, and/ or firmware (e.g., control system 70). The positions ofimaging assembly 33 and stage 50, and the distance between them that areused at initialization can be stored in local or remote memory. Areference specimen, representing a specimen or a specimen classificationgroup can also be used to set parameters specific to the specimen beingexamined. A specimen can be grouped by specimen type, by similarmechanical and/or physical specimen properties (e.g., similarreflectivity properties, similar size dimensions), by feature type, bymanufacturing process and/or examination step, by region of interestand/or any other suitable classification group. Parameters specific tothe specimen being examined can include magnification, focus, adjustinglight exporsure, adjusting illumination (e.g., activating selectedlights and adjusting the light’s intensity level, color and/or angle ofincidence of each selected light). Note, focus can be adjusted bychanging the distance between the specimen and imaging assembly 33,e.g., by mechanism 39 that moves stage 50 in a z direction or by raisingor lowering imaging assembly 33 in a z direction. Exposure can be set byadjusting the camera setting including exposure time, gain, offset,among others. The initial parameters for a specimen can be stored inlocal or remote memory. Example methods for adjusting illumination,include image processing, is described in U.S. Pat. Application No.16/262,017 entitled “Macro Inspection Systems, Apparatus and Methods,”which is hereby incorporated by reference herein in its entirety.

At 720, imaging assembly 33 can be translated either to the right orleft of centerline A2, an amount equal to or greater than the size ofthe artifact appearing in a captured imaged, and a reference image canbe captured. As shown in FIGS. 6A and 6B, imaging artifact 26 appears inthe reference image at the same side of centerline A2 as the position ofimaging assembly 33 when it captured the reference image. The portion ofthe reference image on the opposite side of centerline A2 to imagingassembly 33 will be artifact-free. In some embodiments, imaging assembly33 can be positioned far enough to the right or far enough to the leftof centerline A2, so that in the captured image, imaging artifact 26does not appear exactly at centerline A2, but there is additional space(L and R, respectively, of overlap 118, as shown in FIGS. 6A and 6B)between the imaging artifact and the centerline A2. Before capturing areference image, a specimen can be aligned on stage 50, and imagingassembly 33 positioned in relation to stage 50, so that predeterminededges of a specimen or certain features of a specimen (individually orcollectively, “specimen overlap feature”) fall within an image overlaparea 118 when an image of the specimen is captured by imaging assembly33. The overlap area refers to a predetermined number of pixels (e.g.,1-10 pixels) in the same x, y location of an image captured by imagingassembly 33 when positioned on either side of centerline A2. The shadedareas of bounding box 114 (as shown in FIG. 6A) for image 105 andbounding box 115 (as shown in FIG. 6B) for image 106 represent anartifact-free area that is the same (or overlapping) in images capturedby an imaging assembly when positioned to the right or the left ofcenterline A2. The overlapping areas of a reference image and a secondimage can be used to compare the two images and to select a second imagethat is most similar to the reference image with respect tomagnification, focus and/or exposure. The reference image and the imageselected to be most similar can be digitially stitched (also referred toas “stitched” or “stitching”) together to form a composite artifact-freeimage of the specimen (e.g., image 120 as shown in FIG. 6C). As shown inFIG. 6C, A3 is a line selected in the overlap area.

At 730, imaging assembly 33 is translated to the other side ofcenterline A2, opposite to its position at step 720. In someembodiments, imaging assembly 33 is positioned far enough, to createadditional space between imaging artifact 26 and centerline A2 (e.g., Ror L of image overlap area 118).

At 740, once imaging assembly 33 is properly aligned, imaging assembly33 captures an image of the specimen that includes an imaging artifacton the same side of centerline A2 as the position of imaging assembly 33when capturing the image, and the image is artifact-free on the oppositeside of the centerline. The overlap area 118 of the captured image canbe compared to the overlap area of the reference image, and changes canbe made to the focus, exposure, illumination and/or other parameters, sothat the focus, exposure, illumination and/or other parameters of thereference image and the captured images are the same, or substantiallysimilar. The comparison of overlap areas 118 can be performed manuallyor automatically by software, hardware, and/ or firmware (e.g., bycontrol system 70 and/or computer analysis system 75). Imaging assembly33 can continue to capture images of a specimen until it is determinedthat a captured image matches the focus, exposure, illumination and/orother parameters of the reference image (the “matching image”). Theportions of the reference image and the matching image that include theimaging artifacts can be cropped out of the images, so that theremaining portions are atifact-free (e.g., the area bounded by R1 and R2in FIG. 6A, and the area bounded by R3 and R4 in FIG. 6B).

At 750, an artifact-free portion of thmatching image can be stitchedtogether with an artifact-free portion of the reference image to form acomposite artifact-free image of the specimen (e.g., image 120, as shownin FIG. 6C) Stitching can be performed by aligning the specimen overlapfeature that appears in overlap area 118 of the reference image with thespecimen overlap feature that appears in overlap area 118 of thematching image. Control system 70 and/or computer analysis system 75 cancompare the overlapping areas of each image and digitally adjust thepositions of the images so that the specimen overlap features arealigned. In other embodiments, where a specimen and imaging assembly 33have been precisely aligned so that the overlapping areas of thecaptured images include the same features, stiching can be based on theexact location (e.g., the x/y coordinates) of the overlap area. In someembodiments, the artifact-free portions of the reference image and thematching image can be stitched together directly, without using anyoverlapping areas for alignment.

In another embodiment, imaging assembly 33 can remain fixed and specimenstage (i.e., centerline A2) can be translated to the left or right ofimaging assembly 33 (i.e., optical centerline A1) an offset amountgreater than or equal to the size of the artifact as discussed above.FIG. 8 shows an example imaging method 800 for creating an artifact-freeimage of a specimen by translating specimen stage 50.

Similar to the method described in connection with FIG. 7 , the sameprocess can be repeated (e.g., step 810 (initialize parameters for macroinspection tool and specimen being examined); step 840 (compare thereference image and a second image; if the images do not match, thenmake suitable adjustments to macro inspection system 100 until amatching image is captured) and 850 (crop out the portions of thematching image and the reference image that include the imagingartifacts and stitch together the artifact-free images)), except imagingassembly 33 remains fixed, and stage 50 is moved to the right or left ofoptical centerline A1 to capture images of a specimen on each side ofthe optical centerline (steps 820 and 830). Further, similar to themethod described in connection with FIG. 7 , steps 840 and 850 can beperformed by the control system 70 and/or computer analysis system 75.

FIGS. 9A and 9B, along with FIG. 10 , show embodiments of macroinspection system 100 and an imaging method for creating anartifact-free image of a specimen by using two imaging assemblies 68 and69 and a translatable aperture slider 65.

As shown in FIGS. 9A and 9B, macro inspection system 100 can include:(i) two imaging assemblies 68 and 69, which are positioned, so thattheir optical centerlines B1 and B2 respectively, are offset from and onopposite sides of centerline A2; and (ii) a translatable aperture slider65 having a single opening, aperture 66. The offset amount must besufficient, so that the entire imaging artifact, or the entire imagingartifact and some additional space, will appear in a captured image onone side of specimen centerline A2 (i.e., on the same side as theimaging assembly taking the image).

In this configuration, the imaging assemblies and the specimen stageremain fixed, while aperture slider 65 can be translated in a linearmotion to position aperture 66 beneath one imaging assembly (68 or 69)at a time. In FIG. 9A, aperture slider 65 is translated along apertureslider guide rails 67 so that aperture 66 is positioned beneath imagingassembly 68 (to the left of A2), and a portion of the remaining apertureslider 65 blocks imaging assembly 69 from being reflected in an image ofa specimen captured by imaging assembly 68. Likewise, when apertureslider 65 is tranlated so that aperture 66 is positioned beneath imagingassembly 69 to the right of A2 (as shown in FIG. 9B), the remainingportion of aperture slider 65 blocks imaging assembly 68, so thatimaging assembly 68 is not reflected in an image captured by imagingassembly 69. Aperture slider 65 can be controlled manually, orautomatically by software, hardware, and/ or firmware (e.g., controlsystem 70). Further, aperture slider 65 can be designed to have thesmallest possible diameter without obstructing the imaging field ofeither imaging assembly. In additional embodiments, a dome 42 can becoupled to aperture slider 65 and positioned with aperture 66 to diffusethe light reflected from the specimen.

Note, aperture slider 65 can be made of metal, plastic or other materialthat maintains its shape. In some embodiments, slider 65 is as thin aspossible (typically one to five millimeters), so as not to interferewith imaging space 92. If a dome is not attached, slider 65 can beeither a reflective material or a light absorbing material to preventlight from being reflected. Aperture 66 can be an unobstructed openingor fitted with a lens.

The image 105, as illustrated in FIG. 6A, can be captured when aperture66 is positioned beneath imaging assemby 68 located to the left ofcenterline A2 (as shown in FIG. 9A). Imaging artifact 26 appears on theleft side (i.e., the same side where aperture 66 was positioned when theimage was captured). In contrast, the image 106, as illustrated in FIG.6B, can be captured when aperture 66 is positioned beneath imagingassemby 69 located to the right of centerline A2 (as shown in FIG. 9B).Imaging artifact 26 appears on the right side (i.e., the same side whereaperture 66 was positioned when the image was captured).

FIG. 10 shows an example imaging method 1000 for creating anartifact-free image of a specimen by translating an aperture sliderusing embodiments of macro inspection 100 shown in FIGS. 9A and 9B.

Similar to the method described in connection with FIG. 7 , the sameprocess can be repeated (e.g., step 1010 (initialize parameters formacro inspection tool and specimen being examined); step 1040 (comparethe reference image and a second image, and make adjustments to macroinspection system 100 until a matching image is found) and 1050 (cropout the portions of the matching image and the reference image thatinclude the imaging artifacts and stitch together the matching images)),but instead of moving an imaging assembly or a specimen stage toopposite sides of centerline A2, an aperture on a slider is positioned,in turn, beneath each imaging assembly (e.g., steps 1020 and 1030) tocapture images of a specimen and stitch them together into a compositeartifact-free image of the specimen. Further, similar to the methoddescribed in connection with FIG. 7 , steps 1040 and 1050 can beperformed by the control system 70 and/or computer analysis system 75.

FIG. 11 , along with FIG. 12 , show embodiments of macro inspectionsystem 100 and an imaging method for creating an artifact-free image ofa specimen by rotating a specimen stage 150 and/or an imagingtranslation platform 151 of the macro inspection system 100.

As shown in FIG. 11 , imaging assembly 33 and dome 42 (if included) canbe offset from stage 150 (also known as a ⊖ stage) so that thecenterline of a specimen when placed on the stage is to the right (or tothe left) of optical centerline A1. The offset amount must besufficient, so that the entire imaging artifact, or the entire imagingartifact and some additional space, will appear in a captured image onone side of specimen centerline A2 (i.e., on the same side as theimaging assembly taking the image). Instead of translating imagingassembly 33 or stage 150 laterally (as shown in FIGS. 5A and 5B), eitherspecimen stage 150 or imaging translation platform 151 can be rotatedaround the center of rotation, which is located at A2. Note, the centerof rotation does not have to be aligned with the specimen centerline.

FIG. 12 shows an example imaging method 1200 for creating anartifact-free image of a specimen by rotating a specimen stage usingembodiments of macro inspection 100 shown in FIG. 11 .

At 1210, similar to the method described in connection with FIG. 7 , theparameters for macro inspection system 100 and specimen specificparameters are initialized.

At 1220, an image is captured at an initial position of rotatingspecimen stage 150. FIG. 13A shows an example image 160 that can becaptured when specimen stage 150 (as shown in FIG. 11 ) is at an initialposition. Image 160 shows a specimen having a feature X1 on the topright and a feature X2 on the bottom left. Artifact 26 appears on theleft-hand portion of the image. Bounding box 162 indicates the portionof the image that does not include imaging artifact 26. Line A2indicates the centerline of a specimen. Line A2 can also be used tovertically align the artifact-free images of a specimen to be stitchedtogether.

At 1230, the portion of the image that includes imaging artifact 26 canbe cropped out of the image (as shown in FIG. 13B), leaving only theportion of the image within bounding box 162, representing the rightside of a specimen. Note, that that the portion of the specimencontaining feature X2 is within bounding box 162, and bounding box 162also incorporates area on each side of specimen centerline A2 to createan overlap area that can be used for alignment and stitching an entireimage together. Step 1230, in some examples, is performed by controlsystem 70 and/or computer analysis system 75.

At 1240, the stage can be rotated 180°, or other suitable amount, in aclockwise or counter-clockwise direction, from a first position to asecond position. An image of the specimen can be captured in this secondposition. An example image is shown in FIG. 13C. Note, that since thespecimen was rotated, features X1 and X2 now appear on opposite sides totheir original positions, as captured in image 160, shown in FIG. 13A.Bounding box 164 indicates the portion of the specimen within the imagethat does not show imaging artifact 26. Note, that the position of thespecimen containing feature X1 is within the bounding box and that thebounding box incorporates area on each side of specimen centerline A2 tocreate an overlap area. Note, specimen stage 150 can be rotated anamount other than 180°, as long as the rotation is sufficient to capturetwo images that when stitched together will recreate the specimenwithout including imaging artifact 26.

At 1250, the portion of the image that includes imaging artifact 26 iscropped out of the image (as shown in FIG. 13D), leaving only theportion of the image within bounding box 164, representing the left-handportion of the specimen, or the portion of the specimen that includesfeature X1.

At 1260, as shown in FIGS. 13E-13F, cropped image 164 can be digitallyrotated so that feature X1 appears in its original position (on thebottom left). As described in connection with FIG. 7 , the overlappingareas of two images can be compared. If the images do not match, thenthe focus, exposure, illumination and/or other parameters of macroinspection system 100 can be adjusted and new images captured until amatching pair of images are found.

Once matching images are found, cropped image 164 can be stitchedtogether with cropped image 162, as shown in FIG. 13G, to create acomposite image of the specimen without imaging artifact 26 (step 1270).In some examples, steps 1250, 1260, and 1270 are performed by controlsystem 70 and/or computer analysis system 75.

Note, although FIG. 12 describes a method for creating an artifact-freeimage of a specimen by rotating a specimen stage, an artifact-free imageof a specimen can also be created by carrying out a similar process, butrotating an imaging assembly, instead of a specimen stage.

Note, the methods described herein for inspection of reflectivespecimens is not limited to macroscope inspection systems and can alsobe implemented in microscope inspection systems.

FIG. 14 shows at a high level, an example calibration method 1400 forcalibrating macro inspection system to achieve different illuminationlandscapes, in accordance with some embodiments of the disclosed subjectmatter. Illumination landscape refers to the color and/or intensity oflight on a region of interest of a specimen as a result of theactivation and distribution of light from the one or more of theplurality of lights L1 to Ln that is directed towards a specimen. Theillumination landscape can affect the image captured by imaging assembly33. Control system 70 can control the intensity of one or more of theplurality of lights L1 to Ln to provide a desired illumination landscapeon a specimen plane and/or specimen stage. For example, control system70 can control the intensity of one or more of the plurality of lightsL1 to Ln to provide an illumination landscape of uniform intensity on aspecimen plane and/or specimen stage. The type of illumination landscapeprovided can be determined by the specimen type, mechanical and/orphysical properties of a specimen (e.g., specimen size, specimenreflectivity), a specimen feature being examined, a particular stage ofa manufacturing and/or examining process, or some other suitablevariable, individually or in any combination thereof. In someembodiments, calibration method 1300 can use macro inspection system100.

At 1401, control system 70 can initialize macro inspection system 100.In some embodiments, initialization can include determining theconfiguration of lights L1 to Ln of macro inspection system 100 (e.g.,the total number of lights L1 to Ln, the address and location of eachlight, the position of the light deflector, the area of projection foreach light at each possible position (including height and angle) fromthe light source to the region where the light is projected(collectively, “configuration information″(, and storing theconfiguration information in local or remote memory.

Methods to define an area of illumination projected by each light L1 toLn is described in U.S. Pat. Application No. 16/262,017 entitled “MacroInspection Systems, Apparatus and Methods,” which is hereby incorporatedby reference herein in its entirety.

At 1402, a reference specimen with known features and/ormechanical/physical properties (e.g., size, reflectivity) can be placedon a specimen stage. Different combinations of lights L1 to Ln can beactivated at different colors and/or intensities, at different possibledistances and angles (collectively, “light position″( from the lightsource to the region where the light is projected to determine adesirable illumination landscape for the reference specimen (at 1403).In some embodiments, the desirable illumination landscape can bedetermined based on the quality of images captured by imaging assembly33, based on the measured intensity of light reflected off a specimen Sacross each individual pixel or pixel groups of imaging assembly 33,based on quality of images displayed on a display screen and/or anyother suitable metric. In some embodiments, the illumination landscapecan be adjusted by manually activating different combinations of lightsL1 to Ln at different colors and/or intensities and at differentpossible positions until the desired illumination landscape is achieved.In other embodiments, the illumination landscape can be adjusted byprogramming a set of conditions (e.g., using control system 70 andconfiguration information of 1401) to turn on different combinations oflights L1 to Ln at different colors and/or intensities and at differentlight positions until a desired illumination landscape is achieved. Whenthe desired illumination landscape for a reference specimen is achieved,the address (or other identifying information) of the activated lights,the intensity level and color of each selected light, as well asposition information for each selected light, the distance (e.g., alongthe x, y and z axes) between stage and lens 34 and the position ofimaging assembly 33 and a specimen stage is relation to each other(collectively “illumination profile”(, can be stored (at 1404) bycontrol system 70 for future use.

This process to find and store an appropriate illumination profile canbe repeated for different reference specimens representing differentclassification groups-e.g. by specimen type, by similar mechanicaland/or physical specimen properties (e.g., similar reflectivityproperties, similar size dimensions), by feature type, by manufacturingprocess and/or examination stage, by region of interest and/or any othersuitable classification group. This process can also be repeated for thesame reference specimen to find different illumination profiles that areappropriate for different attributes of the specimen (e.g., asdetermined by a specimen’s mechanical or physical properties); differentspecimen features that are being examined; different regions of intereston the specimen and/or the manufacturing/examination process that isbeing examined. In some embodiments, a reference specimen is first putin focus before an illumination profile is calculated. In furtherembodiments, the distance between specimen stage and lens 34 is adjustedto different preset distances and an illumination profile is calculatedfor a reference specimen at each preset distance.

In embodiments where a uniform illumination landscape is desired, areflective specimen that exhibits a uniform reflective background, asdetermined by standard measurement of reflectivity, can be used tocalibrate macro inspection system 100 to provide a uniform illuminationlandscape. A background can be considered uniform if the reflectivity(e.g., as measured across each individual pixel or pixel groups ofimaging assembly 33) does not vary by more than 5% across the entirefield of view of the specimen when viewed on a specimen stage, andpreferably less than 2%. In some embodiments, a reference specimenwithout a uniform reflective background can be used to calibrate macroinspection system 100 to provide a uniform illumination landscape. Whensuch a specimen is used, lens 34 can be used to create a uniformreflective background by defocusing the specimen to blur any foreignobjects and surface irregularities on the specimen to create a moreuniform reflective background. The illumination landscape can beadjusted by activating different combinations of lights L1 to Ln atdifferent colors and/or intensities and at different possible positionsuntil a uniform illumination landscape is achieved. When a uniformillumination landscape is achieved, the address (or other identifyinginformation) of the activated lights, the intensity and color level ofeach selected light, as well as light position information for eachselected light and the distance between a specimen stage and lens 34 canbe stored by control system 70 as an illumination profile that providesuniform illumination for macro inspection system 100, a particularspecimen, a specimen class, a region of interest, a particular stage inthe manufacturing or examining process, and/or for any other suitableclassification group.

It should be understood that at least some of the portions ofcalibration method 1400 described herein can be performed in any orderor sequence not limited to the order and sequence shown in and describedin connection with FIG. 14 , in some embodiments. Also, some portions ofprocess 1400 described herein can be performed substantiallysimultaneously where appropriate or in parallel in some embodiments.Additionally, or alternatively, some portions of process 1400 can beomitted in some embodiments. Calibration process 1400 can be implementedin any suitable hardware and/or software. For example, in someembodiments, calibration process 1400 can be implemented in macroinspection system 100. Note, that calibration process 1400 is notlimited to macroscope inspection systems and can also be implemented ina microscope inspection system.

FIG. 15A shows at a high level, an example method 1500 for illuminatinga specimen using a macro system to achieve a desired illuminationlandscape )″illumination landscape method 1500″(, in accordance withsome embodiments of the disclosed subject matter. In some embodiments,illumination landscape method 1500 can use macro inspection system 100.

At 1501, a specimen to be examined can be placed on a specimen stage. Insome embodiments, the specimen is brought into focus before theillumination landscape provided by macro inspection system 100 isadjusted.

At 1502, according to some embodiments, control system 70 can activateand adjust the intensity, color and/or pitch of lights L1 to Ln, and/orthe distance between the specimen stage and lens 34 according to astored illumination profile that is selected for the specimen. Theillumination profile can be selected manually or automatically based ona computer algorithm that assesses different attributes of the specimen(e.g., as determined by one or more physical and/or mechanicalproperties of a specimen) and/or different goals of the examination andfinds a suitable illumination profile. Methods for selecting a suitableillumination profile are further discussed in connection with FIG. 14 .

In some embodiments, after selected lights L1 to Ln are activated atdifferent colors and/or intensity, and the selected lights, andadjustments are made to the intensity, color and/or light position,and/or the distance between specimen stage and lens 34, according to aselected illumination profile, further adjustments can be to modify theselected illumination profile to achieve a desired illuminationlandscape. In some embodiments, one or more lights L1 to Ln can beactivated and adjustments can be made to the intensity, color and/orposition of the lights, and/or the distance between a specimen stage andlens 34 without reference to any illumination profile. The activationsand/or adjustments can be performed manually or automatically.

Once one or more of lights L1 to Ln are activated, and adjustments aremade to their intensity, color and/or light position, as well as to thedistance between a specimen stage and lens 34, one or more images of thespecimen can be captured and stored for analysis, as at 1503. In someembodiments, the captured specimen images are transmitted to computeranalysis system 75.

At 1505, a determination is made by computer analysis system 75 as towhether the applied activation of one or more of lights L1 to Ln, andadjustments to their intensity, color and/or light position, etc. aresufficient to produce a desired illumination landscape. Suchdeterminations may be made based on an analysis of pixel intensityvalues for image data received during the image capture step of 1503. Ifthe illumination landscape profile is determined to be sub-optimal, thenprocess 1500 can revert back to step 1502, and further adjustments tothe illumination landscape can be made. Steps 1502-1505 can iterateuntil an optimal illumination profile is achieved. By way of example, ifan illumination landscape with a uniform light intensity profile isdesired for a particular specimen type, but the image data associatedwith the captured one or more specimen images indicate that some regionsare insufficiently illuminated, then step 1505 can revert back to step1502. In step 1502, additional changes to light activation, intensity,position (elevation and/or pivot/rotation), etc. can be made. Oncechanges have been applied to the illumination landscape, step 1503 isrepeated and image data is collected from the specimen under the newconditions, e.g., by an image capture device. Again, at step 1505, thenew illumination landscape is analyzed to determine if optimal lightingconditions have been achieved.

Different illumination profiles can be selected for a specimen, and foreach selected illumination profile, control system 70 can activate andadjust the intensity, color and/or position of lights L1 to Ln, and/ordistance between a specimen stage and lens 34 according to the selectedprofile, and capture and store one or more images of the specimen. Assuch, the iterative process of steps 1502-1505 can differ with specimentype, as the initially applied illumination landscape that is applied atstep 1502 may vary with specimen type, region of interest, a particularstage in the manufacturing or examining process, and/or for any othersuitable classification group. In some embodiments, once theillumination is configured according to a selected illumination profile,at step 1507, a specimen stage and/or imaging assembly 33 can beadjusted to different positions in relation to each other and one ormore images of the specimen can be captured at each position.

FIG. 15B illustrates steps of an example process 1510 for identifying aspecimen classification and automatically adjusting an illuminationlandscape of the macro inspection apparatus, according to some aspectsof the disclosed technology. Process 1510 begins with step 1512 whichimage data is received, for example, by an image processing system e.g.,image processing system 1634, discussed above. In some approaches, theimage data can be included in a received image of a specimen that istaken by an imaging device, as part of macro inspection system 100. Theimage data can include all or a portion of a specimen that is disposedon a stage of macro inspection system 100. In some instances, that imagedata may only comprise pixel intensity values, indicating an intensityof light reflected from different portions of a specimen surface.

In step 1514, the image data is analyzed to identify a classification ofthe specimen. In some instances image analysis may be performed toidentify a subset of the specimen, such as a particular region orfeature. As discussed below, machine learning classifiers, computervisions and/or artificial intelligence can be used to identify/classifythe specimen.

Subsequently, an illumination profile can be automatically selectedbased on the specimen (or feature) classification and/or a particularstage in the manufacturing or examining process. The specimen/featureclassification can be used to query an illumination profile databasethat contains one or more illumination profiles associated with specimenand/or specimen feature types. By referencing the specimenclassification determined in step 1514, a matching illumination profilecan be automatically identified and retrieved. As discussed above, theillumination profile can contain a variety of settings data thatdescribe configurations of macro inspection system 100 that can be usedto achieve the optimal illumination landscape for the specimen orfeature being observed.

It should be understood that at least some of the portions ofillumination landscape method 1500 described herein can be performed inany order or sequence not limited to the order and sequence shown in anddescribed in connection with FIGS. 15A and 15B, in some embodiments.Also, some portions of process 1300 described herein can be performedsubstantially simultaneously where appropriate or in parallel in someembodiments. Additionally, or alternatively, some portions of process1500 can be omitted in some embodiments. Illumination landscape method1500 can be implemented in any suitable hardware and/or software. Forexample, in some embodiments, illumination landscape method 1500 can beimplemented in macro inspection system 100. Note, that illuminationlandscape method 1500 is not limited to macroscope inspection systemsand can also be implemented in microscope inspection systems.

FIG. 16 shows the general configuration of an embodiment of computeranalysis system 75, in accordance with some embodiments of the disclosedsubject matter. Although computer analysis system 75 is illustrated as alocalized computing system in which various components are coupled via abus 1605, it is understood that various components and functionalcomputational units (modules) can be implemented as separate physical orvirtual systems. For example, one or more components and/or modules canbe implemented in physically separate and remote devices, such as, usingvirtual processes (e.g., virtual machines or containers) instantiated ina cloud environment.

Computer analysis system 75 includes a processing unit (e.g., CPU/sand/or processor/s) 1610 and bus 1605 that couples various systemcomponents including system memory 1615, such as read only memory (ROM)1620 and random access memory (RAM) 1625, to processor/s 1610.

Memory 1615 can include various memory types with different performancecharacteristics. Processor 1610 is coupled to storage device 1630, whichis configured to store software and instructions necessary forimplementing one or more functional modules and/or database systems,such as profile generation module 1632, illumination profile database1636, and imaging processing module 1634. Each of these modules can beconfigured to control processor 1610 as well as a special-purposeprocessor where software instructions are incorporated into the actualprocessor design. As such, processor 1610 and one or more of profilegeneration module 1632, illumination profile database 1636, and imagingprocessing module 1634 can be completely self-contained systems. Forexample, imagine processing module 1634 can be implemented as a discreteimage processing system, without departing from the scope of thedisclosed technology.

To enable user interaction with computer analysis system 75, inputdevice 1645 can represent any number of input mechanisms, such as amicrophone for speech, a touch-sensitive screen for gesture or graphicalinput, keyboard, mouse, motion input and so forth. An output device 1635can also be one or more of a number of output mechanisms known to thoseof skill in the art. In some instances, multimodal systems can enable auser to provide multiple types of input to communicate with computeranalysis system 75, for example, to convey specimen information relatingto a specimen type/classification, or other characteristics.Communications interface 1640 can generally govern and manage the userinput and system output. There is no restriction on operating on anyparticular hardware arrangement and therefore the basic features heremay easily be substituted for improved hardware or firmware arrangementsas they are developed.

Storage device 1630 is a non-transitory memory and can be a hard disk orother types of computer readable media that can store data accessible bya computer, such as magnetic cassettes, flash memory cards, solid statememory devices, digital versatile disks, cartridges, random accessmemories (RAMs) 1525, read only memory (ROM) 1520, and hybrids thereof.

In practice, illumination profile generation module 1632 can beconfigured to receive a scan of a specimen, or a portion of a specimen(collectively, “specimen image”(, from macro inspection system 100,and/or any suitable computer readable media. In some instances,preferred illumination landscapes associated with configurations of thevarious macro components of macro inspection system 100 can beassociated to form an illumination profile, for example, that isassociated with the specimen type or classification. Illuminationprofiles associating illumination landscape settings with specimenclassification types can be stored to illumination profile database1636.

Illumination profiles stored to illumination profile database 1636 caninclude specific context data such as: a configuration of lights L1 toLn of macro inspection system 100 (e.g., the total number of lights L1to Ln, the address and location of each light, the position of lightdeflector 83, the area of projection for each light at each possibleposition that it can be located (including height and angle) from thelight source to the region where the light is projected); the range ofpossible distances between specimen stage and lens 34; the range ofdifferent positions of specimen stage and imaging assembly 33 inrelation to each other, regions of interest for particular types ofspecimen; a particular stage of a manufacturing or examining processthat is being examined; a feature that is being examined.

Image processing system 1634 can be used in conjunction with profilegeneration module 1632 and illumination profile database 1636 toclassify a specimen based on the image data received in the specimenimage(s) and/or other received specimen characteristics, such as thosemanually provided by a user, for example, via input device 1645.Additionally, image processing system can be configured to classifyspecific specimen features, determine other physical and/or mechanicalspecimen properties (e.g., specimen reflectivity, specimen dimensions).Classifications of specimen types, and specimen features/properties canbe stored as part of an illumination profile. As such, variousillumination profiles stored in illumination profile database 1636 cancontain settings and parameters used to generate an optimal illuminationlandscape that can be referenced and matched to a sample based on sampletype and or specific features or characteristics.

In some aspects, classification of a specimen type and/or features of aspecimen can be performed using image processing algorithms that caninclude computer vision, one or more artificial intelligencealgorithm(s) and/or computer algorithms. Classification of a specimen,or features of a specimen, can also be based on, e.g., a computer aideddesign (CAD) file of a specimen and/or features of a specimen, aspecimen layout map identifying features on a specimen, images of knownspecimens and/or features, and/or information about known specimens(e.g., a specimen’s dimensions, the mechanical and/or physicalproperties of a specimen).

In some instances, machine learning models can be used to performclassification of specimens, specimen features, and/or other specimencharacteristics. In some aspects, image data from specimen images can beprovided as an input to a machine learning classification system, forexample, by image processing system 1634. Classifier output can specifya sample or feature classification that can then be used to reference anillumination profile stored in illumination profile database 1636. Bymatching the correct illumination profile with the correct sampleclassification or feature type, the correct illumination landscape canbe achieved through the automatic calibration of light intensity, lightcolor, lighting angle, and elevation above the specimen, etc.

As understood by those of skill in the art, machine learning basedclassification techniques can vary depending on the desiredimplementation, without departing from the disclosed technology. Forexample, machine learning classification schemes can utilize one or moreof the following, alone or in combination: hidden Markov models;recurrent neural networks; convolutional neural networks; Bayesiansymbolic methods; general adversarial networks; support vector machines;image registration methods; applicable rule-based system. Whereregression algorithms are used, they may include including but are notlimited to: a Stochastic Gradient Descent Regressor, and/or a PassiveAggressive Regressor, etc.

Machine learning classification models can also be based on clusteringalgorithms (e.g., a Mini-batch K-means clustering algorithm), arecommendation algorithm (e.g., a Miniwise Hashing algorithm, orEuclidean LSH algorithm), and/or an anomaly detection algorithm, such asa Local outlier factor. Additionally, machie learning models can employa dimensionality reduction approach, such as, one or more of: aMini-batch Dictionary Learning algorithm, an Incremental PrincipalComponent Analysis (PCA) algorithm, a Latent Dirichlet Allocationalgorithm, and/or a Mini-batch K-means algorithm, etc.

Such algorithms, networks, machines and systems provide examples ofstructures used with respect to any “means for determining anillumination profile for a specimen using artificial intelligence.”

In some embodiments, machine learning can be deployed in the creation ofillumination profiles. For example, profile generation module 1632 caninput the context data, along with the specimen image or data determinedfrom the specimen image )″specimen data″( into a trained artificialintelligence algorithm to create one or more appropriate illuminationprofiles to be applied to illuminate a specimen. In other embodiments,image processing system 1634 can use machine learning models or othercomputer algorithms to select a predefined illumination profile based onthe specimen image, specimen data and/or context data, as discussedabove.

Once the desired illumination profile has been selected, e.g., fromillumination profile database 1636, the illumination profile data can betransmitted to control system 70. Control system 70 can use thisinformation in connection with process 1400 to apply an illuminationprofile to illuminate a specimen being examined.

Examples of artificial intelligence based image processing algorithmthat can be used by illumination profile generation module 1632 is imageregistration as described by: Barbara Zitova, “Image RegistrationMethods: A Survey,” Image and Vision Computing, Oct. 11, 2003, Volume21, Issue 11, pp. 977-1000, which is hereby incorporated by referenceherein in its entirety. The disclosed methods are just examples and arenot intended to be limiting.

In some embodiments, the machine learning algorithms used byillumination profile generation module 1632, and image processing system1634, including, in some embodiments, an image processing algorithm, isfirst trained with training data so that illumination profile generationmodule 1632 can create an appropriate illumination profile for aspecimen.

As shown in FIG. 17 , training data 1701 can include labeled images ofknown specimens and features captured by a macro inspection systemaccording to embodiments of the disclosed subject. The labeled imagesselected for training can be images of desired quality that showsuitable detail based on an inspection objective for the capturedimages. In some embodiments, training data 1701 can include non-imagefiles identifying the type of specimen and/or features being inspected.Training data can further include for each image: data describing theactivation, intensity, color, position data for (i) lights L1 to Ln;(ii) the distance (along the x, y and z axes) between a specimen stageand lens 34; and (iii) the position of the speciman stage and imagingassembly 33 in relationship to each other, the features of a specimenbeing inspected; the region of interest on the specimen being inspected;the particular stage of a manufacturing or examining process beinginspected. In some embodiments training data can includephysical/mechanical properties of a specimen, and/or any other suitablecharacteristic used to create an appropriate illumination profile. Insome embodiments, training data can also include unlabeled data.

Once the artificial intelligence algorithm used by illumination profilegeneration module 1632 is trained, it can be applied by illuminationprofile generation module 1632 to a received specimen scan to create oneor more illumination profiles (output data 1702) for each receivedspecimen image. As described above, illumination profile data caninclude data identifying which lights L1 to Ln to activate, and at whatintensity, color and light position. Illumination profile data can alsoinclude a distance (e.g., along the x, y and z axis) between a specimenstage and lens 34, as well as the position of the speciman stage andimaging assembly 33 in relation to each other.

Note that macro inspection system 100 can include other suitablecomponents not shown. Additionally or alternatively, some of thecomponents included in macro inspection system 100 can be omitted.

In some embodiments, any suitable computer readable media can be usedfor storing instructions for performing the functions and/or processesdescribed herein. For example, in some embodiments, computer readablemedia can be transitory or non-transitory. For example, non-transitorycomputer readable media can include media such as non-transitorymagnetic media (such as hard disks, floppy disks, etc.), non-transitoryoptical media (such as compact discs, digital video discs, Blu-raydiscs, etc.), non-transitory semiconductor media (such as flash memory,electrically programmable read only memory (EPROM), electricallyerasable programmable read only memory (EEPROM), etc.), any suitablemedia that is not fleeting or devoid of any semblance of permanenceduring transmission, and/or any suitable tangible media. As anotherexample, transitory computer readable media can include signals onnetworks, in wires, conductors, optical fibers, circuits, and anysuitable media that is fleeting and devoid of any semblance ofpermanence during transmission, and/or any suitable intangible media.

The logical operations of the various embodiments are implemented as:(1) a sequence of computer implemented steps, operations, or proceduresrunning on a programmable circuit within a general use computer, (2) asequence of computer implemented steps, operations, or proceduresrunning on a specific-use programmable circuit; and/or (3)interconnected machine modules or program engines within theprogrammable circuits. The macroscopic inspection system of the subjectinvention can practice all or part of the recited methods, can be a partof the recited systems, and/or can operate according to instructions inthe recited non-transitory computer-readable storage media. Such logicaloperations can be implemented as modules configured to control theprocessor to perform particular functions according to the programmingof the module.

The various systems, methods, and computer readable mediums describedherein can be implemented as part of a cloud network environment. Asused in this paper, a cloud-based computing system is a system thatprovides virtualized computing resources, software and/or information toclient devices. The computing resources, software and/or information canbe virtualized by maintaining centralized services and resources thatthe edge devices can access over a communication interface, such as anetwork. The cloud can provide various cloud computing services viacloud elements, such as software as a service (SaaS) (e.g.,collaboration services, email services, enterprise resource planningservices, content services, communication services, etc.),infrastructure as a service (IaaS) (e.g., security services, networkingservices, systems management services, etc.), platform as a service(PaaS) (e.g., web services, streaming services, application developmentservices, etc.), and other types of services such as desktop as aservice (DaaS), information technology management as a service (ITaaS),managed software as a service (MSaaS), mobile backend as a service(MBaaS), etc.

The provision of the examples described herein (as well as clausesphrased as “such as,” “e.g.,” “including,” and the like) should not beinterpreted as limiting the claimed subject matter to the specificexamples; rather, the examples are intended to illustrate only some ofmany possible aspects. A person of ordinary skill in the art wouldunderstand that the term mechanism can encompass hardware, software,firmware, or any suitable combination thereof.

Unless specifically stated otherwise as apparent from the abovediscussion, it is appreciated that throughout the description,discussions utilizing terms such as “determining,” “providing,”“identifying,” “comparing” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system memories or registersor other such information storage, transmission or display devices.Certain aspects of the present disclosure include process steps andinstructions described herein in the form of an algorithm. It should benoted that the process steps and instructions of the present disclosurecould be embodied in software, firmware or hardware, and when embodiedin software, could be downloaded to reside on and be operated fromdifferent platforms used by real time network operating systems.

The present disclosure also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored on acomputer readable medium that can be accessed by the computer. Such acomputer program may be stored in a computer readable storage medium,such as, but is not limited to, any type of disk including floppy disks,optical disks, CD-ROMs, magnetic-optical disks, read-only memories(ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic oroptical cards, application specific integrated circuits (ASICs), or anytype of non-transient computer-readable storage medium suitable forstoring electronic instructions. Furthermore, the computers referred toin the specification may include a single processor or may bearchitectures employing multiple processor designs for increasedcomputing capability.

The algorithms and operations presented herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may also be used with programs in accordancewith the teachings herein, or it may prove convenient to construct morespecialized apparatus to perform the required method steps andsystem-related actions. The required structure for a variety of thesesystems will be apparent to those of skill in the art, along withequivalent variations. In addition, the present disclosure is notdescribed with reference to any particular programming language. It isappreciated that a variety of programming languages may be used toimplement the teachings of the present disclosure as described herein,and any references to specific languages are provided for disclosure ofenablement and best mode of the present disclosure.

It is understood that any specific order or hierarchy of steps in theprocesses disclosed is an illustration of exemplary approaches. Basedupon design preferences, it is understood that the specific order orhierarchy of steps in the processes may be rearranged, or that only aportion of the illustrated steps be performed. Some of the steps may beperformed simultaneously. For example, in certain circumstances,multitasking and parallel processing may be advantageous. Moreover, theseparation of various system components in the embodiments describedabove should not be understood as requiring such separation in allembodiments, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.

The system, method and apparatus for macroscopic inspection ofreflective specimen have been described in detail with specificreference to these illustrated embodiments. It will be apparent,however, that various modifications and changes can be made within thespirit and scope of the disclosure as described in the foregoingspecification, and such modifications and changes are to be consideredequivalents and part of this disclosure. The scope of the presentdisclosure is limited only by the claims that follow.

Statements of this disclosure include:

Statement 1. An inspection apparatus, comprising: a specimen stageconfigured to retain a specimen, one or more imaging devices positionedabove the specimen stage to capture one or more images of the specimenfrom the specimen stage; a set of lights disposed on a platform betweenthe specimen stage and the imaging device; and a control system coupledto the specimen stage, the one or more imaging devices, and theplatform, the control system comprising: one or more processors; andmemory storing executable instructions that, as a result of beingexecuted by the one or more processors, cause the control system to:provide first instructions to the one or more imaging devices to capturea first image of the specimen, the first image comprising a firstimaging artifact to a first side of a reference point; provide secondinstructions to the one or more imaging devices to capture a secondimage of the specimen, the second image comprising a second imagingartifact to a second side of the reference point; crop the first imagingartifact from the first image and the second imaging artifact from thesecond image; and digitally stitch together the first image and thesecond image to generate a composite image of the specimen, thecomposite image lacking the first imaging artifact and the secondimaging artifact.

Statement 2. An inspection apparatus according to Statement 1, whereinthe executable instructions further cause the control system to:translate the one or more imaging devices in a first direction to afirst position above and to the first side of the reference point tocapture the first image of the specimen; and translate the one or moreimaging devices in a second direction to a second position above and tothe second side of the reference point to capture the second image ofthe specimen.

Statement 3. An inspection apparatus according to any of Statements 1and 2, wherein the executable instructions further cause the controlsystem to: translate the specimen stage to a first position under and tothe first side of the reference point to capture the first image of thespecimen; and translate the specimen stage in a second direction underand to the second side of the reference point to capture the secondimage of the specimen.

Statement 4. An inspection apparatus according to any of Statements 1through 3, wherein the reference point is positioned along a centerlineof the specimen.

Statement 5. An inspection apparatus according to any of Statements 1through 4, wherein the imaging device is moveable along a rotationalaxis.

Statement 6. An inspection apparatus according to any of Statements 1through 5, wherein: the one or more imaging devices include: a firstimaging device positioned above and over to the first side of thereference point; and a second imaging device positioned above and overto the second side of the reference point; and the inspection apparatusfurther comprises an aperture slider positioned below the first imagingdevice and the second imaging device, the aperture slider comprising anaperture to allow capture of images of the specimen using either thefirst imaging device or the second imaging device.

Statement 7. An inspection apparatus according to Statement 6, wherein:the control system translates the aperture slider to a first positionsuch that the aperture is aligned with the first imaging device tocapture the first image; and the control system translates the apertureslider to a second position such that the aperture is aligned with thesecond imaging device to capture the second image.

Statement 8. An inspection apparatus according to any of Statements 1through 7, wherein the executable instructions further cause the controlsystem to: translate the platform; activate one or more combinations ofthe set of lights to determine an illumination profile; analyze thefirst image of the specimen to identify a specimen classification;select, based on the specimen classification, the illumination profile;and adjust the platform and the set of lights according to theillumination profile.

Statement 9. An inspection apparatus according to any of Statements 1through 8, further comprising a barrier configured to diffuse lightreflected from the specimen retained on the specimen stage back onto thespecimen.

Statement 10. An inspection apparatus according to any of Statements 1through 9, wherein the executable instructions further cause thecontrolling system to compare a first overlap area of the first image toa second overlap area of the second image to determine that a matchingimage has been identified to allow for digital stitching of the firstimage and the second image.

Statement 11. A method, comprising: receiving a specimen on a specimenstage of an inspection apparatus; identifying a reference point for theinspection system; capturing a first image of the specimen, the firstimage comprising a first imaging artifact to a first side of thereference point; capturing a second image of the specimen, the secondimage comprising a second imaging artifact to a second side of thereference point; evaluating the second image of the specimen todetermine that the second image is usable with the first image togenerate a composite image of the specimen; cropping the first imagingartifact from the first image and the second imaging artifact from thesecond image; and digitally stitching together the first image and thesecond image to generate the composite image of the specimen, thecomposite image lacking the first imaging artifact and the secondimaging artifact.

Statement 12. A method according to Statement 11, wherein the methodfurther comprises: translating an imaging device of the inspectionapparatus in a first direction to a first position above and to thefirst side of the reference point to capture the first image; andtranslating the imaging device in a second direction to a secondposition above and to the second side of the reference point to capturethe second image.

Statement 13. A method according to any of Statements 11 and 12, whereinthe method further comprises: translating the specimen stage in a firstdirection to a first position under and to the first side of thereference point to capture the first image; and translating the specimenstage to a second position under and to the second side of the referencepoint to capture the second image.

Statement 14. A method according to any of Statements 11 through 13,wherein the method further comprises: rotating the specimen stage to afirst position to capture the first image; cropping the first image toremove a first portion of the first image, the first portion includingthe first imaging artifact; rotating the specimen stage to a secondposition to capture the second image; cropping the second image toremove a second portion of the second image, the second portionincluding the second imaging artifact; and digitally rotating the secondimage to initiate evaluation of the second image.

Statement 15. A method according to any of Statements 11 through 14,wherein the method further comprises: translating an aperture slider ofthe inspection apparatus in a first direction to position an aperturebelow a first imaging device of the inspection apparatus to capture thefirst image, the first imaging device being positioned above and to thefirst side of the reference point; and translating the aperture sliderof the inspection apparatus in a second direction to position theaperture below a second imaging device of the inspection apparatus tocapture the second image, the second imaging device being positionedabove and to the second side of the reference point.

Statement 16. A method according to any of Statements 11 through 15,further comprising: translating a platform of the inspection system,wherein a set of lights are disposed on the platform; activating one ormore combinations of the set of lights to determine an illuminationprofile; analyzing the first image of the specimen to identify aspecimen classification; selecting, based on the specimenclassification, the illumination profile; and adjusting the platform andthe set of lights according to the illumination profile.

Statement 17. A method according to any of Statements 11 through 16,further comprising: rotating an imaging device of the inspectionapparatus in a first direction to position the imaging device to thefirst side of the reference point to capture the first image; androtating the imaging device of the inspection apparatus in a seconddirection to position the imaging device to the second side of thereference point to capture the second image.

Statement 18. A method according to any of Statements 11 through 17,further comprising: diffusing light reflected from the specimen retainedon the specimen stage back onto the specimen.

Statement 19. A method according to any of Statements 11 through 18,wherein the method further comprises: comparing a first overlap area ofthe first image to a second overlap area of the second image todetermine that a matching image has been identified to allow for digitalstitching of the first image and the second image.

Statement 20. A method according to any of Statements 11 through 19,wherein the specimen stage is moveable along an X axis, a Y axis, a Zaxis, and a rotational axis.

1. A method, comprising: generating, by a computing system, a trainingdata set for training a machine learning model to generate anillumination profile for illuminating a specimen under examination in amacro inspection system, the training data set comprising images ofknown specimens and illumination profile data corresponding to theimages of known specimens; training, by the computing system, themachine learning model to generate illumination profiles forilluminating specimens under examination based on the training data set,wherein the machine learning model learns features of the knownspecimens and correlates the learned features with the illuminationprofile data; determining, by the computing system, that the machinelearning model has achieved a threshold level of accuracy in generatingthe illumination profiles for illuminating specimens under examination;and based on the determining, deploying, by the computing system, themachine learning model in a macro inspection environment.
 2. The methodof claim 1, wherein the training data set further comprises: non-imagedata identifying known specimens and features of the known specimens. 3.The method of claim 1, wherein the illumination profile data in thetraining data set comprises one or more of an activation value, anintensity value, or a color value of each light utilized forexamination.
 4. The method of claim 1, wherein the training data setfurther comprises: a region of interest for each of the known specimens.5. The method of claim 1, wherein the training data set furthercomprises: an indication of a particular stage of manufacturing for theknown specimen undergoing the examination.
 6. The method of claim 1,wherein training, by the computing system, the machine learning model togenerate the illumination profiles comprises: training the machinelearning model to predict one or more of an activation value, anintensity value, or a color value of each light utilized forexamination.
 7. The method of claim 1, wherein the training data setcomprises: a distance of the known specimen to a lens of the macroinspection system.
 8. A non-transitory computer readable mediumcomprising one or more sequences of instructions, which, when executedby one or more processors, causes a computing system to performoperations comprising: generating, by the computing system, a trainingdata set for training a machine learning model to generate anillumination profile for illuminating a specimen under examination in amacro inspection system, the training data set comprising images ofknown specimens and illumination profile data corresponding to theimages of known specimens; training, by the computing system, themachine learning model to generate illumination profiles forilluminating specimens under examination based on the training data set,wherein the machine learning model learns features of the knownspecimens and correlates the learned features with the illuminationprofile data; determining, by the computing system, that the machinelearning model has achieved a threshold level of accuracy in generatingthe illumination profiles for illuminating specimens under examination;and based on the determining, deploying, by the computing system, themachine learning model in a macro inspection environment.
 9. Thenon-transitory computer readable medium of claim 8, wherein the trainingdata set further comprises: non-image data identifying known specimensand features of the known specimens.
 10. The non-transitory computerreadable medium of claim 8, wherein the illumination profile data in thetraining data set comprises one or more of an activation value, anintensity value, or a color value of each light utilized forexamination.
 11. The non-transitory computer readable medium of claim 8,wherein the training data set further comprises: a region of interestfor each of the known specimens.
 12. The non-transitory computerreadable medium of claim 8, wherein the training data set furthercomprises: an indication of a particular stage of manufacturing for theknown specimen undergoing the examination.
 13. The non-transitorycomputer readable medium of claim 8, wherein training, by the computingsystem, the machine learning model to generate the illumination profilescomprises: training the machine learning model to predict one or more ofan activation value, an intensity value, or a color value of each lightutilized for examination.
 14. The non-transitory computer readablemedium of claim 8, wherein the training data set comprises: a distanceof the known specimen to a lens of the macro inspection system.
 15. Asystem comprising: a processor; and a memory having programminginstructions stored thereon, which, when executed by the processor,causes a computing system to perform operations comprising: generating,by the computing system, a training data set for training a machinelearning model to generate an illumination profile for illuminating aspecimen under examination in a macro inspection system, the trainingdata set comprising images of known specimens and illumination profiledata corresponding to the images of known specimens; training, by thecomputing system, the machine learning model to generate illuminationprofiles for illuminating specimens under examination based on thetraining data set, wherein the machine learning model learns features ofthe known specimens and correlates the learned features with theillumination profile data; determining, by the computing system, thatthe machine learning model has achieved a threshold level of accuracy ingenerating the illumination profiles for illuminating specimens underexamination; and based on the determining, deploying, by the computingsystem, the machine learning model in a macro inspection environment.16. The system of claim 15, wherein the training data set furthercomprises: non-image data identifying known specimens and features ofthe known specimens.
 17. The system of claim 15, wherein theillumination profile data in the training data set comprises one or moreof an activation value, an intensity value, or a color value of eachlight utilized for examination.
 18. The system of claim 15, wherein thetraining data set further comprises: a region of interest for each ofthe known specimens.
 19. The system of claim 15, wherein the trainingdata set further comprises: an indication of a particular stage ofmanufacturing for the known specimen undergoing the examination.
 20. Thesystem of claim 15, wherein training, by the computing system, themachine learning model to generate the illumination profiles comprises:training the machine learning model to predict one or more of anactivation value, an intensity value, or a color value of each lightutilized for examination.